In recent years, Mixed Reality (MR) that aims at seamlessly coupling a real space and virtual space has been extensively studied. MR has received a lot of attention as a technique which aims at realizing coexistence of a virtual reality (VR) world that can be expressed in only a situation isolated from a real space, and the real space, and augments VR.
A typical apparatus for implementing MR is a head-mounted display (HMD). More specifically, MR is implemented by mixing and displaying real and virtual spaces on the HMD. MR schemes using the HMD include an optical see-through scheme for superposing a CG image or the like onto an image displayed on a semi-transparent (see-through) HMD, and a video see-through scheme for mixing a CG image or the like onto image data sensed by a video camera attached to an HMD, and then displaying the mixed image on the HMD.
MR can be used, quite possibly, in new fields which are qualitatively quite different from VR, such as a medical assist application for presenting the state inside the body of a patient to a doctor as if it were seen through, a job assist application for superposing and displaying the assembly sequence of a product on real parts in a factory, and the like.
A technique for removing “deviation” between real and virtual spaces is commonly required for these applications. “Deviation” can be classified into positional deviation, temporal deviation, and qualitative deviation, and many studies have been conventionally made for removal of positional deviation as the most fundamental requirement among those deviations.
Especially, in case of video see-through MR, since an image process scheme can be relatively easily applied as a method of correcting positional deviation, alignment using an image process has been conventionally proposed.
More specifically, a method in which markers which are marked in color so an image process can easily detect are laid out at a predetermined position in a real space, and the viewpoint position is computed on the basis of the marker positions detected from an image sensed by a camera attached to a player, and a method of correcting an output signal from a location/posture sensor based on marker positions in an image, and the like are available.
When markers in image data are detected, and the location/posture of a player is estimated based on the detection result, the markers must appear in an image to have an appropriate size and nearly uniform intervals. Also, since a sufficient number of markers must be simultaneously detected in the image upon computing the viewpoint position, the markers must be laid out to be observed in the image at somewhat narrow intervals.
On the other hand, in order to improve the tracking or identification precision of markers, the markers must be laid out to be observed in the image at somewhat broad intervals.
In case of a single player, it is not so difficult to lay out markers to satisfy the aforementioned conditions. However, in an application which allows a plurality of players to share a common MR space, markers which are laid out at equal intervals at positions that can be observed from a given player cannot often be observed at equal intervals from another player.
For this reason, Japanese Patent Laid-Open No. 11-84307 has proposed an arrangement in which in an air hockey game system in which two players hit and return a virtual puck toward each other on a table as a real object, markers having different colors in units of players are provided to allow the individual players to observe the markers with a preferred layout and size.
However, marker layout using different colors becomes harder with increasing number of players who share a single MR space. More specifically, in order to detect a given color by an image process, the colors of the markers and background object, and those of markers in units of users must be easily detected and extracted by the image process. However, when the number of colors used increases, it becomes difficult to satisfy such conditions, and extraction errors and identification errors among markers can occur.
FIG. 10 is a graph for explaining recognition error factors when a plurality of different color markers are used. In FIG. 10, the abscissa plots red, and the ordinate plots green. For the sake of simplicity, a blue axis is not shown. In FIG. 10, region A defines the color distribution of marker type A (red marker), and region B defines the color distribution of marker type B (orange marker). In this manner, when the number of players increases and similar colors are used as markers, even though a player observes a red marker, the marker may be detected as color of region B (i.e., of orange marker), thus causing detection errors.
When a plurality of markers having different colors in units of players are laid out, a very large number of markers appear in the real space, and may complicate the vision of the player, thus impairing reality upon experiencing an MR space.